Automatic model training under restrictive time constraints

Abstract We develop a hyperparameter optimisation algorithm, Automated Budget Constrained Training, which balances the quality of model with computational cost required to tune it. The relationship between hyperparameters, and must be learnt this learning is incorporated directly into problem. At each training epoch, algorithm decides whether terminate or continue training, and, in latter case, what values hyperparameters use. This decision weighs optimally potential improvements additional time uncertainty about quantities. performance our verified on number machine problems encompassing random forests neural networks. Our approach rooted theory Markov processes partial information we numerical method compute value function an optimal strategy.